Inspired by the selectiveinference framework, we propose an approach that enables learning the hyperparameters and testing onthefullsample without data splitting.

At the same significance level, our approach's test power is empirically

The kernel Maximum Mean Discrepancy (MMD) is a popular multivariate distance metric between distributions that has found utility in two-sample testing.

We establish that testing for the equality of two probability measures on a general separable and compact metric space is equivalent to testing for the singularity between two corresponding Gaussian measures on a suitable Reproducing Kernel Hilbert Space. The corresponding Gaussians are defined via the notion of kernel mean and covariance embedding of a probability measure. Discerning two singular Gaussians is fundamentally simpler from an information-theoretic perspective than non-parametric two-sample testing, particularly in high-dimensional settings. Our proof leverages the Feldman-Hajek criterion for singularity/equivalence of Gaussians on Hilbert spaces, and shows that discrepancies between distributions are heavily magnified through their corresponding Gaussian embeddings: at a population level, distinct probability measures lead to essentially separated Gaussian embeddings. This appears to be a new instance of the blessing of dimensionality that can be harnessed for the design of efficient inference tools in great generality.

Machine learning and deep learning have been used extensively to classify physical surfaces through images and time-series contact data. However, these methods rely on human expertise and entail the time-consuming processes of data and parameter tuning. To overcome these challenges, we propose an easily implemented framework that can directly handle heterogeneous data sources for classification tasks. Our data-versus-data approach automatically quantifies distinctive differences in distributions in a high-dimensional space via kernel two-sample testing between two sets extracted from multimodal data (e.g., images, sounds, haptic signals). We demonstrate the effectiveness of our technique by benchmarking against expertly engineered classifiers for visual-audio-haptic surface recognition due to the industrial relevance, difficulty, and competitive baselines of this application; ablation studies confirm the utility of key components of our pipeline. As shown in our open-source code, we achieve 97.2% accuracy on a standard multi-user dataset with 108 surface classes, outperforming the state-of-the-art machine-learning algorithm by 6% on a more difficult version of the task. The fact that our classifier obtains this performance with minimal data processing in the standard algorithm setting reinforces the powerful nature of kernel methods for learning to recognize complex patterns.

Nonparametric two-sample testing, aiming to determine whether two collections of samples are from the same distribution without specifying the exact parametric forms of the distributions, is one of the fundamental problems in statistics. Such tests have found applications in various areas such as bioinformatics, anomaly detection, model criticism, audio and image processing. The most traditional tools in this domain include the Kolmogorov-Smirnov test [Smirnov (1939)], Cramér-von Mises test [Anderson (1962)], Wald-Wolfowitz runs test [Wald and Wolfowitz (1940)], and Wilcoxon-Mann-Whitney test [Mann and Whitney (1947)]. Extensions and generalizations of these tests were studied in Darling (1957), Weiss (1960), Bickel (1969), Friedman and Rafsky (1979), among many others. Modern nonparametric tests have been developed based on integral probability metrics [Sriperumbudur et al. (2012)]. Notable members include the kernel Maximum Mean Discrepancy (MMD) two-sample test [Gretton et al. (2012)] and energy distance-based two-sample test [Székely et al. (2004)]. These metrics are gaining increasing popularity in both the statistics and machine learning communities and they have been applied to a plethora of statistical problems including goodness-of-fit testing [Székely and Rizzo (2005)], nonparametric analysis of variance [Rizzo and Székely (2010)], change-point detection [Matteson and James (2014); Chakraborty and Zhang (2021)], finding representative points of a distribution [Mak and Joseph (2018)] and controlled variable selection [Romano et al. (2020)].

Kernel two-sample tests have been widely used and the development of efficient methods for high-dimensional large-scale data is gaining more and more attention as we are entering the big data era. However, existing methods, such as the maximum mean discrepancy (MMD) and recently proposed kernel-based tests for large-scale data, are computationally intensive to implement and/or ineffective for some common alternatives for high-dimensional data. In this paper, we propose a new test that exhibits high power for a wide range of alternatives. Moreover, the new test is more robust to high dimensions than existing methods and does not require optimization procedures for the choice of kernel bandwidth and other parameters by data splitting. Numerical studies show that the new approach performs well in both synthetic and real world data.

Evaluating whether data streams were generated by the same distribution is at the heart of many machine learning problems, e.g. to detect changes. This is particularly relevant for data generated by dynamical systems since they are essential for many real-world processes in biomedical, economic, or engineering systems. While kernel two-sample tests are powerful for comparing independent and identically distributed random variables, no established method exists for comparing dynamical systems. The key problem is the critical independence assumption, which is inherently violated in dynamical systems. We propose a novel two-sample test for dynamical systems by addressing three core challenges: we (i) introduce a novel notion of mixing that captures autocorrelations in a relevant metric, (ii) propose an efficient way to estimate the speed of mixing purely from data, and (iii) integrate these into established kernel-two sample tests. The result is a data-driven method for comparison of dynamical systems that is easy to use in practice and comes with sound theoretical guarantees. In an example application to anomaly detection from human walking data, we show that the test readily applies without the need for feature engineering, heuristics, and human expert knowledge.

Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is nontrivial in practice. Although kernel selection for two-sample test has been studied, the insufficient samples in change point detection problem hinders the success of those developed kernel selection algorithms. In this paper, we propose KL-CPD, a novel kernel learning framework for time series CPD that optimizes a lower bound of test power via an auxiliary generative model. With deep kernel parameterization, KL-CPD endows kernel two-sample test with the data-driven kernel to detect different types of change-points in real-world applications. The proposed approach significantly outperformed other state-of-the-art methods in our comparative evaluation of benchmark datasets and simulation studies. Detecting changes in the temporal evolution of a system (biological, physical, mechanical, etc.) in time series analysis has attracted considerable attention in machine learning and data mining for decades (Basseville et al., 1993; Brodsky & Darkhovsky, 2013). This task, commonly referred to as change-point detection (CPD) or anomaly detection in the literature, aims to predict significant changing points in a temporal sequence of observations.

We show how to obliviously embed into the reproducing kernel Hilbert space associated with Gaussian kernels, so that distance in this space (the kernel distance) only has $(1+\varepsilon)$-relative error. This only holds in comparing any point sets at a kernel distance at least $\alpha$; this parameter only shows up as a poly-logarithmic factor of the dimension of an intermediate embedding, but not in the final embedding. The main insight is to effectively modify the well-traveled random Fourier features to be slightly biased and have higher variance, but so they can be defined as a convolution over the function space. This result provides the first guaranteed algorithmic results for LSH of kernel distance on point sets and low-dimensional shapes and distributions, and for relative error bounds on the kernel two-sample test.